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1 

l Adjusting an Electronic Camera to Acquire a Watermarked Image 

2 

3 Field of the Invention: 

4 This invention relates to electronic cameras and more particularly to the control of 

5 electronic cameras. 

6 

7 Background of the Invention: 

8 Electronic cameras are commercially available and in widespread use. The basic 

9 technology used in electronic cameras is well developed. Electronic cameras generally 

10 include CCD or CMOS light sensing elements. The exposure time of the light sensing 

1 1 elements and the gain of the readout circuitry can be set to control the quality of the 

12 images which are acquired by the camera. 
13 

14 Some electronic cameras have a built in exposure meter that detects the amount of light 

15 reaching the camera. The appropriate exposure time and gain of the sensing elements is 

16 then automatically set at values determined by the signal generated by the exposure 

17 meter. 
18 

19 Some electronic video cameras which are designed for integration into a personal 

20 computers provide the user with a menu such as that shown in Figure 1 which can be 

21 used to set various parameters to control the operation of the camera. A user typically 

22 sets the various parameters so that the camera will produce an image that is visually 

23 appealing to a human observer. A program in the computer adjusts the actual settings in 

24 the camera in a direction which attempts to have the images meet the specifications set 

25 by the user. This type of adjustment may take place over several sequential frames. 

26 Furthermore as ambient conditions change, the actual exposure time and gain of the 

27 sensing elements is varied between frames in an effort to meet the specifications set by 

28 the user. 
29 

30 Electronic cameras can be used as part of systems such as that shown in copending 

31 applications 09/314.648 : 09/343,104, 60/158,015, and 60/163,332. The systems shown 

32 in the referenced copending applications are designed to detect a digital watermark in an 

33 image acquired by a video camera. In such systems after a video camera is used to 

34 acquire an image, the acquired image is digitally processed to detect a digital watermark 
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1 that is contained in the image. It has been found that the optimal settings for a camera 

2 used in such systems is not necessarily the optimal settings that are established to 

3 provide a pleasing image to a human viewer. 

4 

5 The type of parameters that can be established using a menu such as that shown in 

6 Figure 1 do not always produce optimal conditions for detecting a digital watermark in an 

7 acquired image. 
8 

9 Summary of the present Invention: 

10 The present invention provides a mechanism for automatically setting the exposure time 

1 1 and gain of an electronic camera so that the acquired image has improved characteristics 

12 for the detection of a digital watermark. With the present invention the exposure time and 

13 gain of an electronic camera are set by first directing the camera at an image with the 

14 camera's controls set to a set of default or initial values. The system then calculates and 

15 stores data which represents a histogram of the luminance values of the pixels in the 

16 acquired image. Next characteristic values from the histogram data such as the peak 

17 value, the centroid value, the RMS value, the median value, etc., are calculated. One or 

18 more of the characteristic values of the histogram data are then applied as an index or 

19 pointer to values of pre-established settings that are stored in a table. Values from the 

20 table are used to set the exposure time and/or gain of the camera. In some 

21 embodiments, the variance of the luminance histogram data is also measured and used to 

22 index data from the table. In some embodiments, a test target is used to acquire an initial 

23 image. In other embodiments, the initial image is made by pointing the camera at the 

24 image which contins the watermark data. 

25 

26 Detailed Description of the Figures: 

27 Figure 1 illustrates the screen presented to a user to enable the user to set the values that 

28 control a camera. 
29 

30 Figure 2 is an overall diagram of a first embodiment of the present invention. 
31 

32 Figure 3 is a diagram that illustrates a luminance histogram. 
33 



WO 01/39106 



PCTYUS00/32013 



3 

1 Figure 4 is a diagram illustrating a table which stores values for exposure time for a 

2 camera's shutter. 

3 

4 Figure 5 is a flow diagram of the control program executed by the computer embedded in 

5 the camera. 
6 

7 Figure 6 is a flow diagram of an alternative embodiment. 
8 

9 Detailed Description of Embodiments of the invention: 

10 The overall system shown in Figure 2 includes a conventional digital camera 21 and a 
] ] conventional, control computer 22. As is conventional the camera 21 includes a CCD 

12 detector 23 and a lens 24 which focuses an image on the CCD detector. The CCD 

13 detector 23 has a gain control line 23a and an exposure control line 23b. The control 

14 computer 22 controls the values on the lines 23a and 23b. While for convenience of 

15 illustration and explanation the camera 21 is shown in Figure 2 as an individual unit 

16 separated from control computer 22, it should be understood that in many commercially 

17 available systems the camera 21 is mounted within (or attached to) the computer 22 and 

18 lines 23a and 23b represent logical control lines rather than physical wires . 
19 

20 As is conventional the computer 22 includes memory 22a and programming 22b. For 

21 convenience and clarity of illustration and explanation these are illustrated as separate 

22 boxes in Figure 2. The other conventional parts of the camera 21 and the computer 22 

23 are not illustrated in Figure 2 and not discussed herein since they are not particularly 

24 relevant to an explanation of the invention. 
25 

26 In the first preferred embodiment of the present invention, the camera 21 is first directed 

27 at a test target with the exposure time and gain of the camera set to the default values. 

28 When the camera is activated, the camera will detect an image, each pixel of which has a 

29 certain luminance value. The luminance value of each pixel is a particular numeric value 

30 between 0 and 255. The number of pixels at each luminance value is counted. The coun 

31 values form a histogram such as that shown in Figure 3. Naturally in computer 22. the 

32 histogram is stored as data values. Figure 3 is merely an illustration of an example of 

33 such .data. 
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1 The test target used in the first preferred embodiment is a standard photographer's "gray 

2 card". That is : a card that has an image consisting of a gray background that is eighteen 

3 percent gray. 

4 

5 Various mathematical parameters can be calculated from a histogram (or from the data 

6 that can form a histogram). In first the preferred embodiment, the centroid value of the 

7 luminance data is calculated and used for subsequent operations. However, it is noted 

8 that in alternative embodiments ! other values are used including: 



9 a) The peak value of the histogram, 

10 b) The root-mean-square value of the points that form the histogram, 
l ] c) The median value of the points that form the histogram. 

12 d) The deviation value of the curve. 

13 e) etc. 

14 



15 The value of a parameter from the histogram data (in the first preferred embodiment that 

16 is the centroid value) is used as an index or pointer to values in a table which contains 

17 exposure values. An example of such a table is given in Figure 3. Naturally it should be 

18 understood that in computer 22, the data is stored as a conventional data table. Figure 4 

19 is merely an illustration of such a table. 
20 



21 Figure 5 is a flow diagram illustrating the steps in the operation of the system shown in 

22 Figure 2. First, as indicated by block 501 , the control parameters of the camera are set to 

23 a set of default values. In particular the exposure time is set to 1/90 second. Next as 

24 indicated by block 502, the camera is pointed at a test target and an image is acquired. 

25 The test target is a conventional photographers gray scale that is eighteen percent gray. 
26 

27 The luminance of the pixels in the image are measured (as indicated by block 503) and a 



28 histogram of the luminance values is generated. In the particular embodiment, each pixel 

29 has a luminance between 0 and 255 (i.e. an 8 bit representation) and the histogram has 

30 an appearance somewhat like the histogram shown in Figure 3. Naturally, the actual 

3 1 values will be dependent on various factors such as the amount of ambient light present. 

32 Furthermore, if a 16 bit system were used, the luminance would have values between 0 

33 and 65535. 
34 
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1 Next (as indicated by block 504) the centroid of the histogram is calculated in a 

2 conventional manner. That is, the centroid is calculated from the luminance data directly. 

3 There is no need to actually construct or calculate the histogram. The histogram is merely 

4 discussed herein to principles of the invention. A centroid is calculated directly from the 

5 luminance data by adding together each luminance value times the number of pixels at 

6 that value. For example: assume that there are: 

7 3 pixels with value 1, 

8 2 pixels with value 2 

9 6 pixels with value 3 

10 5 pixels with value 4 

11 etc. . 

12 The centroid is then calculated as: 

13 (3x1 + 2x2 + 6x3 +5x4 + etc.) / (3+2+6+5 ..-.etc. ) 

14 The value calculated for the centroid is used as an index or pointer to select a value in a 

15 table such as the table shown in Figure 4 as indicated by block 506). 
16 

17 Next (block 506) the exposure time for the camera is set to the value obtained from the 

18 table and an image is acquired using that exposure time (block 507). Finally (block 508) 

19 a watermark is read from the acquired image in a conventional manner. 

20 

21 A flow diagram for a second preferred embodiment of the invention is shown in Figure 6. 

22 In this alternative embodiment, no test target is used. Instead, the camera is pointed at 

23 the image which contains a watermark and repeated frames containing the image are 

24 • acquired. Between each frame a new setting for the camera is calculated using 

25 luminance values as previously described; however, in this embodiment, the luminance 

26 values are the luminance values of an image that contains a watermark. After each 

^ image is acquired an attempt is made to read the watermark, if the watermark can not be 

28 read, the new camera setting are used to acquire another image. The process repeats 

29 until the watermark can be read. 
30 

3 1 Figure 6 is a flow diagram of the operations. The camera is set to a relatively slow frame 

32 rate (such as five frames per second) so that there is sufficient time between frames to 

33 process the data to detect a watermark. This operation is conventional. The calculations 
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1 to determine a new setting for the camera take much less time than do the calculations to 

2 detect a watermark. 

3 



4 The process begins as indicated by block 601 when the camera is pointed at an image 

5 containing a watermark and an image is acquired. As indicated by blocks 602A and 

6 602B, two operations proceed in parallel. First an attempt is made to read the watermark. 

7 If that is successful (block 603) the operation goes to block 605 where the watermark data 

8 is used for purposes such as those described in the previously referenced co-pending 

9 applications. The second program that operates after each image is acquired in the 

10 program indicated by block 602B. This program assembles data that forms a histogram of 

] ] the luminance of the pixels in the image, the centroid is calculated and a new exposure 

12 value is obtained from a table (similar to the operation of the previous embodiment). Next 

13 the camera is set to the new exposure value as indicated by block 604. 



14 

15 Finally if the watermark reading operation was not successful another image is acquired 

16 and the process repeats. 

17 

18 The following is a more a specific description of a program that implements the operations 

19 shown in Figure 6. This embodiment of the invention utilizes a commercially available 

20 "3Com HomeConnecf PC Digital Camera. The camera as commercially available 



21 provides a software interface, which allows many parameters to be queried and modified. 

22 The available parameters and their value ranges shown in parenthesis are: 

23 

24 - Enable/disable the interface (e/d) 

25 - Enable/disable Auto White Balance (AWB) tracking (e/d) 

26 - Red balance (0-255) 

27 - Blue balance (0-255) 

28 - Color saturation (0-255) 

29 - Set the mode of Auto Gain Control (AGC) operation (disabled, average mode ; 

30 average center window mode, peak mode) 

31 - AGC bias (0-100) 

32 - Shutter value (0-3900) 

33 - Video gain (0-255) 

34 - Gamma value (0-255) 

35 - Image sharpness (0-255) 
36 

37 The only parameter that the program which implements the present invention changes 



38 dynamically in its automatic operation is the shutter value. This value relates to a shutter 
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1 speed and has a non-linear range of 0-3900. In this scale, 0 is a shutter speed (or 

2 exposure) of 1/31 ,500 of a second and 3900 is a shutter speed of Vi of a second. 

3 



4 When the program starts, the first thing it does is to enable the software interface to the 

5 camera. The program saves the current values of all the camera parameters and then 

6 puts in its own set of values. The camera's initial values will be restored when the 

7 program shuts down. The following are initial values for various parameters that have 

8 been found to operate satisfactorily in this particular embodiment of the invention. 

9 - AWB: disabled 

10 - Red balance: 225 

11 - Blue balance: 128 

12 - Color saturation: 128 

13 - AGC: disabled 

14 - Video gain: 128 

15 - Shutter: 174 

16 - Gamma: 100 



17 

18 When each image is acquired the program starts examining the video frame for 

19 watermarks as indicated by block 602A. As each frame is received, its luminance centroid 

20 is calculated and compared to our target luminance. The program dynamically adjusts the 

21 shutter value in order to move the centroid towards the target luminance as indicated by 

22 block 604. 
23 

24 Values used in the algorithm: 



25 - Target luminance: 110 (range is 0-255) 

26 - Distance is the distance between Centroid and Target 

27 - Midpoint is one half of Distance 

28 - Direction flag: set to 1 if Centroid < Target, otherwise set to -1 

29 - LastShutter is the shutter value from the last frame we received 

30 - Shutter is the current shutter value 

3 1 - LastLuminance is the centroid for the last frame we received 

32 - DeltaLuminance is the difference between LastLuminance and Centroid 

33 - AdjustmentAmount is the change that we will be making to the Shutter value 

34 for this frame 

35 - LastAdjustment is the AdjustmentAmount we made on the last frame we 

36 received 

37 - Adjustment table (defined below) 
38 



39 The adjustment table lays out ranges of luminance values along with adjustment factors 

40 and a scale flag. The adjustment factor is a multiplier which is used along with the 

41 distance from Centroid to Midpoint. If the scale flag is set for a particular range then the 

42 Current Shutter value is also factored into calculations involving that range. 
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1 

2 Lower bound Upper bound Factor Flag 

3 0 29 7.5 Off 

4 30 69 0.75 On 

5 70 94 0.95 On 

6 95 104 0.75 On 

7 105 115 0.5 On 

8 116 135 0.5 On 

9 136 150 0.7 On 

10 151 225 1.5 On 

11 226 255 2.0 Off 
I2 

13 The following algorithm is implemented by the program: 

14 

15 If Centroid has not changed since the last frame and the distance from Centroid to 

16 Target < 20 then don't make any adjustments 

17 

18 Set CurrentAdjustment to 0 
19 

20 Walk through the Adjustment Table. For each entry in the table, if any part of the 

21 Area, defined as the area between Centroid and 

22 Midpoint falls within the range of Lower Bound to Upper Bound then calculate 

23 Scale and Adjustmentlncrement as follows: 

24 

25 If Flag is Off then 

26 Scale = Factor 

27 Else 

28 Scale = Factor * Log (Current Shutter) 

29 

30 Adjustmentlncrement = ( amount of Area that falls between Lower Bound 

31 and Upper Bound ) * Scale * Direction 

32 

33 Each Adjustmentlncrement is added to CurrentAdjustment. 

34 
35 

36 Compare DeltaLuminance to LastAdjustment. If DeltaLuminace is a positive 

37 number while LastAdjustment is negative or vice-versa then it means that 

38 movement of Centroid is going in the opposite direction from our adjustments. We 

39 will continue to adjust until Centroid movement switches directions, then we will 

40 slow down and wait for Centroid to catch up. At the same time we will limit 

41 CurrentAdjustment changes to be no more than 25% of the Shutter value. 

42 

43 AdjustmentPercentage = Abs( CurrentAdjustment / Shutter ) 
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1 If AdjustmentPercentage > 25% 

2 CurrentAdjustment = Shutter * 25% * Direction 

3 

4 Else 

5 

6 CentroidRate = Abs( DeltaLuminance / LastLuminance ) 

7 

8 If LastAdjustment was non-zero and CurrentAdjustment is non-zero 

9 

10 ShutterRate = Abs( LastAdjustment / LastShutter ) 

1 1 AdjustmentRatio = Abs( CurrentAdjustment / LastAdjustment ) 
12 

13 If ShutterRate > CentroidRate and ShutterRate < AdjustmentRatio 

14 

15 AdjustmentPercentage = Abs( CurrentAdjustment / Shutter ) 

16 MaxPercentage = ( Target - Distance ) / Target 
17 

18 If AdjustmentPercentage > MaxPercentage 

19 

20 CurrentAdjustment = Shutter * MaxPercentage * 

21 Direction 

22 

23 Else 

24 

25 TestAdjustment = CurrentAdjustment * 3 * 

26 CentroidRate / ShutterRate 

27 

28 If TestAdjustment < CurrentAdjustment 

29 

30 CurrentAdjustment = TestAdjustment 

31 

32 

33 Check to make sure that CurrentAdjustment will not push Shutter below 0. If so 

34 then 

35 

36 CurrentAdjustment = Shutter * 2/3 ^Direction 

37 
38 

39 Make the change then update our values: 

40 

4 1 Shutter = Shutter + CurrentAdjustment 

42 

43 LastShutter = Shutter 

44 LastCentroid = Centroid 

45 LastAdjustment = CurrentAdjustment 

46 
47 



48 
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1 It is noted that in the preferred embodiments described above, the invention is used to 

2 only set the exposure time of the camera. In other embodiments, the table shown in 

3 Figure 4 is expanded to also include the value of the gain and the gain of the camera is 

4 set utilizing the present invention. In such an embodiment both exposure time and gain 

5 can be set using one parameter from the histogram data. Various rules could be used to 

6 construct the table used to obtain values of the exposure time and gain, that is, to map the 

7 one dimensional parameter value into a two dimensional exposure/gain value. In similar 

8 fashion other parameters of the camera can be set utilizing the present invention. 
9 

10 While the invention has been described with respect to preferred embodiments thereof, it 

l ] should be understood that various other changes in form and detail are possible without 

12 departing from the sprit and scope of the invention. For example, while the invention is 

13 particularly illustrated with reference to a camera employing a CCD detector, any other 

14 optical sensor technology can likewise be employed. Similarly, while the invention is 

15 particularly described with reference to a video camera, the same principles are equally 

16 applicable to watermark detection with non-video image acquisition device. Yet further, 

17 while a table was used in the illustrated embodiment to generate camera settings, other 

18 known structures and approaches (e.g., formulas) using the same or similar input 

19 information can be readily substituted. Still further, while a histogram was described as a 

20 preferred analysis technique, other statistical analyses for characterizing the camera's 

21 response can similarly be utilized. Accordingly, the scope of applicant's invention is 

22 limited only by the structure and methods described in the appended claims and 

23 equivalents thereto. 

24 

25 I claim: 
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1 1 ) The method of adjusting the control setting of an electronic camera which comprises: 

2 adjusting said control setting to a default value, 

3 acquiring a first image with the camera, 

4 the luminance values of the pixels in said first image forming a histogram, 

5 calculating the value of at least one parameter of said histogram. 

6 using said value to obtain a second value for said control setting, 

7 adjusting said camera in accordance with said second value, 

8 acquiring at least one additional image with said control setting set to said second value. 

9 

10 2) The method claimed in claim 1 including the step of reading a digital watermark from 

] i said additional image. 

12 

13 3) The method claimed in claim 1 wherein said parameter is the centroid of said 

14 histogram. 
15 

16 4) The method claimed in claim 3 including the step of reading a digital watermark from 

17 said additional image. 
18 

19 5) The method in claim 1 wherein said parameter is the peak value of said centroid. 
20 

21 6) The method in claim 1 wherein said control setting is the exposure time of said camera. 

22 

23 7) The method in claim 1 wherein the first image acquired by said camera is an image of a 

24 photographer's gray card. 

25 

26 8) The method in claim 1 wherein said parameter is the variance of said histogram. 

27 

28 9) The method in claim 1 wherein control setting includes the gain of said camera. 
29 

30 10) An electronic camera which has at least one control which can be adjusted: 

31 means for adjusting said control setting to a default value, 

32 means for acquiring a first image with the camera, 

33 the luminance values of the pixels in said first image forming a histogram, 

34 means for calculating the value of at least one parameter of said histogram, 
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1 means for using said value to obtain a second value for said control setting, 

2 means for adjusting said camera in accordance with said second value, 

3 means for acquiring at least one additional image with said control setting set to said 

4 second value. 

5 

6 1 1) An electronic camera which has at least one control which can be adjusted: 

7 a computer which controls said camera, 

8 a program for controlling said computer to facilitate, 

9 adjusting said control to a default value, 

10 acquiring a first image with the camera with said control adjusted to said default value, 

1 1 the luminance values of the pixels in said first image forming a histogram, 

12 calculating the value of at least one parameter of said histogram, 

13 using said value to obtain a second value for said control setting, 

14 adjusting the control of said camera in accordance with said second value, 

15 acquiring at least one additional image with said control setting set to said second value, 

16 detecting a watermark in said additional image. 

17 

18 12) The camera recited in claim 1 1 wherein said parameter is the centroid of said 

19 histogram. 
20 

21 13) The camera recited in claim 1 1 wherein said parameter is the peak value in said 

22 histogram. 

23 

24 14) The camera recited in claim 11 wherein said control is the exposure time of said 

25 camera. 

26 

27 15) The method recited in claim 1 wherein said first image is an image of a 

28 photographer's gray card. 
29 

30 16) The method recited in claim 1 wherein said default setting is an exposure time of 

31 1/90 second. 
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1 

2 17) The method of adjusting the control setting of an electronic camera which comprises: 

3 acquiring a first image with the camera, 

4 the luminance values of the pixels in said first image forming a histogram, 

5 calculating the value of at least one parameter of said histogram, 

6 using said value to obtain a new value for said control setting, 

7 adjusting said camera in accordance with said new value ; 

8 attempting to read a watermark in said first image, and 

9 if said read operation is not successful, repeating said process. 
10 

11 18) The method recited in claim 1 7 wherein said parameter is the centroid of said 

12 histogram. 
13 

14 19) The method recited in claim 17 wherein said control setting is the exposure time of 

15 said camera. 

16 
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